Having specified the factor model, we want to know how much of the variability in $\mathbf X$, given by the covariance matrix $\mathbf \Sigma$, where $\mathbf \Sigma=\text{Cov}(\mathbf X) = (\mathbf X- \mathbf M)(\mathbf X- \mathbf M)^T$, is explained by the factor model.
Suppose there are $p$ original variables, to be explained by $m$ factors ($m<p$). Factor analysis decomposes the $p \times p$ variance-covariance matrix $\mathbf \Sigma$ of the original variables $\mathbf X$ into a $p \times m$ loadings matrix $\mathbf \Lambda$, where $\mathbf \Lambda= \text{Cov}(XF)$, and a $p \times p$ diagonal matrix of unexplained variances of original variables, $\mathbf \Psi$, where $\mathbf \Psi = \text{Cov}(\mathbf E)$, such that
$$\mathbf \Sigma = \mathbf \Lambda \mathbf \Lambda^T+ \mathbf \Psi$$This equation indicates that we know the variability in $\mathbf X$, given by $\mathbf \Sigma$, if we know the loadings matrix $\mathbf \Lambda$ and the diagonal matrix of unexplained variances $\mathbf \Psi$. Thus, more conceptual, we explain the $\mathbf \Sigma$ by two terms. The first term, the loadings matrix $\mathbf \Lambda$ gives the coefficients $(\lambda_{jm})$ that relate the factors $(F_{jm})$ to each particular observations $(x_j)$. These coefficients, as we will discuss in the subsequent sections, may be estimated given the observational data. Consequently, the term $\mathbf\Lambda \mathbf\Lambda^T$ corresponds to the variability, which may be explained by the factors. This proportion of the overall variability, explained be a linear combination of factors, is denoted as communality. In contrast, the proportion of variability, which can not be explained by a linear combination of the factors, given by the term $\mathbf \Psi$ is denoted as uniqueness.
$$\mathbf \Sigma = \underbrace{\mathbf\Lambda\mathbf\Lambda^T}_{\text{communality}} + \underbrace{\mathbf\Psi}_{\text{uniqueness}}$$Citation
The E-Learning project SOGA-Py was developed at the Department of Earth Sciences by Annette Rudolph, Joachim Krois and Kai Hartmann. You can reach us via mail by soga[at]zedat.fu-berlin.de.
Please cite as follow: Rudolph, A., Krois, J., Hartmann, K. (2023): Statistics and Geodata Analysis using Python (SOGA-Py). Department of Earth Sciences, Freie Universitaet Berlin.